On Reachability of Markov Chains: A Long-Run Average Approach
نویسندگان
چکیده
We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of suitable long-run average reward problem, we study various reachability/controllability problems. First, characterize domain attraction escape set system, generalization called $p$-domain attraction, using aforementioned function. Next, solve problem maximizing probability reaching notation="LaTeX">$A$ while avoiding notation="LaTeX">$B$. Finally, constrained version previous where ask for notation="LaTeX">$B$ to be bounded. In finite case, use linear programming formulations these
منابع مشابه
Reachability problems for Markov chains
We consider the following decision problem: given a finite Markov chain with distinguished source and target states, and given a rational number r, does there exist an integer n such that the probability to reach the target from the source in n steps is r? This problem, which is not known to be decidable, lies at the heart of many model checking questions on Markov chains. We provide evidence o...
متن کاملMarkov Decision Processes with Multiple Long-Run Average Objectives
We consider Markov decision processes (MDPs) with multiple long-run average objectives. Such MDPs occur in design problems where one wishes to simultaneously optimize several criteria, for example, latency and power. The possible trade-offs between the different objectives are characterized by the Pareto curve. We show that every Pareto optimal point can be ε-approximated by a memoryless strate...
متن کاملReachability in Augmented Interval Markov Chains
In this paper we propose augmented interval Markov chains (AIMCs): a generalisation of the familiar interval Markov chains (IMCs) where uncertain transition probabilities are in addition allowed to depend on one another. This new model preserves the flexibility afforded by IMCs for describing stochastic systems where the parameters are unclear, for example due to measurement error, but also all...
متن کاملThe Long Run Impact of Technology Diffusion on Average Cost in Upstream Oil Industry; Case Stud of Iran
Literature review related to nonrenewable resources shows that technological improvements have considerable effects on resource depletion and decreasing operational cost. Therefore it is assumed that technology is the most important and influential variables in the production function and utilization cost of these resources. In this study, we assess the long term effect of technology diffusion...
متن کاملOn Differentiability of Average Cost in Parameterized Markov Chains
The purpose of this appendix is to prove Theorem 4.6 in [5] and establish various facts used in verifying some of the assumptions of Theorem A.1 therein. We use the same notation and assumptions as in [5], and to simplify the analysis, we assume that the parameter θ is scalar. The setting used here is similar (but not identical) to the one in [4]. The major difference is that the setting of [4]...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Automatic Control
سال: 2022
ISSN: ['0018-9286', '1558-2523', '2334-3303']
DOI: https://doi.org/10.1109/tac.2021.3071334